1. Technical Field of the Invention
The present invention relates to image sensor noise reduction and particularly, but not exclusively, to reduction of photon shot noise in image sensors without increasing image sensor size.
2. Description of Related Art
Image sensors are typically constructed of a matrix of pixels. Unwanted variations in pixel output, or noise, is a significant problem in image sensors. There are many noise sources and the dominating factor depends on the particular situation.
For low light levels, the noise in dark scenes is important and is typically cause by one of dark current (leakage through the photodiode and its variations); readout noise (thermal noise of the transistors); and reset noise, also known as kTC noise, which comes from uncertainties in resetting the capacitance on the photodiode.
For high light levels, photon shot noise dominates the noise sources. Photon shot noise results from natural fluctuations in the number of photons detected by a pixel: a photon separates electrons from atoms and is therefore “detected” by collecting electrons which have been separated. The randomness in the number of photons detected is manifested as a temporal and spatial fluctuation in the signal produced by the pixel. Photon shot noise occurs even with an ideal noise-free pixel. Thus, photon shot noise imposes a fundamental limit on the responsiveness of a pixel insofar as it determines the minimum noise level achievable.
Photon shot noise is described by the square root of the number of photons hitting a pixel per unit time (or in other words, the square root of the intensity or flux (F) of incident radiation). Therefore increasing flux density (F) reduces the relative fraction of photon shot noise.
The full well capacity of a pixel refers to the total amount of electrons or charge that can be stored in the pixel before overflowing into an adjoining pixel. Since this feature is a result of the pixel value being stored on a capacitor, which is in turn dependent on pixel area, the full well capacity of a pixel is dependent among other things upon its physical size.
As such, the full well capacity of a pixel provides an upper limit on the number of photons that can be integrated. Thus, any attempt to increase the flux density of the radiation incident on a pixel and the detection thereof must be accompanied by an increase in the full well capacity of the pixel.
Typically, for a well designed, modern, high resolution camera, with a 10 bit ADC (Analog to Digital Converter), the noise sources from low light combined is around 0.5-1 ADC units, while the photon shot noise is 8 ADC units.
Unfortunately, the commercial pressure is to reduce pixel size as this either reduces the size of the device, which is important for size-critical applications such as mobile phones, laptop PCs, PDAs and the like, or increased pixel count. Reducing the size of the pixel, reduces the capacitances and hence the full well capacity also reduced, thereby increasing the noise.
For high quality images in average or good lighting systems, the photon shot noise dominates and hence techniques and methods to reduce this are advantageous.
Reducing the photon shot noise usually involves increasing the pixel's full-well capacity. There are several known techniques: increasing the pixel capacitance, often by increasing the size of the pixel (but thereby increasing the size and/or cost of the sensor; increasing the pixel's voltage swing, usually through special manufacturing process (even though modern CMOS sensors are made using an optimized process, full-well capacity is often limited to 10 kē to 20 kē); or special architecture of the pixel, for example “photon counting”. The trouble with these techniques is that, due to the pixel complexity, they are only suitable for larger (20 μm) pixel sizes and not for smaller (<3 μm). That is, the associated circuitry required cannot be arranged within the dimensions of a smaller size pixel.
It is well known that one method to reduce noise in an image is to average several pixels together. Often this is done spatially, in fact many commercial image sensors have high photon shot noise but their resolution (e.g. 3 Mpix-10 Mpix) is much higher than the resolution of the display (screen, projector etc.), so they are “zoomed”, or resealed, to fit which involves averaging pixels together. However, if the image is cropped, examined at its native resolution or requires the full resolution of the sensor, the photon shot noise is readily apparent.
Some image sensors provide on-chip scaling—this can be done at the digital level or even at the pixel level (also known as “analog binning”).
Another method of averaging is temporal averaging. Here several images are acquired, stored and averaged on a per-pixel basis. This maintains the full spatial resolution, but allows for a reduction in noise.
In prior art temporal averaging systems, an image sensor produces image frames at the same rate as the frames are outputted. For example, PAL (phase-alternating line) is a 50 Hz interlaced system with 625 lines, the image sensor in a PAL system produces 50½ vertical resolution images per second (often called a “field”) and these are displayed on a PAL output display, such as a television. These techniques can work at any frame/field rate, for example: 25 Hz, 29.97 Hz, 60 Hz, 59.94 Hz, 100 Hz, 120 Hz, etc.
There are various de-interlacing systems which combine the output, with or without averaging, of two 50 Hz fields (½ vertical resolution) into a single frame (full vertical resolution) at either 50 Hz or 25 Hz rate. However, de-interlacing systems by definition have to interpolate in one manner or another as each field is distinct either spatially or temporally.
For high definition systems, for example 50 Hz/720 p, the sensor produces 50 full (720 lines) images per second and the transmission and display is also 50 images per second, each image having 720 lines of information. As such, no averaging is performed between frames.
There is accordingly a need in the art for an improved manner of reducing image sensor noise.